The present invention relates to an image data coding and/or decoding system for compressing and coding an image data into a smaller amount of data and for decoding code strings obtained by the compression coding to reproduce an image. More specifically, the invention relates to an image data coding and/or decoding system which has high error resilience and which can transmit and/or store a coded image data of high quality even if the coded image data is transmitted and/or stored via a medium wherein an error easily occurs, such as a radio channel.
In an image transmitting and/or storing system such as a video telephone, a videoconferencing system, a portable information terminal, a digital video disc system and a television broadcasting system, various systems, which include the motion compensation, the discrete cosine transform, the sub-band coding, the pyramid coding and the combinations thereof, have been developed as techniques for compressing and coding an image data into a smaller amount of data for transmission/storing. In addition, as international standard methods for compressing and coding a moving picture, the methods xe2x80x9cISOxe2x80xa2MPEG1xe2x80x9d, xe2x80x9cISOxe2x80xa2MPEG2xe2x80x9d, xe2x80x9cITU-Txe2x80xa2H.261xe2x80x9d, and xe2x80x9cITU-Txe2x80xa2H.262xe2x80x9d are standardized. All these methods are compressing and coding methods, each comprising a combination of the motion-compensated adaptive prediction and the discrete cosine transform, and described in detail in xe2x80x9cInternational Standard of Multimedia Codingxe2x80x9d (edited and written by Hiroshi Yasuda, published by Maruzen, June 1991) (Literature 1) and so forth.
As an example of conventional moving-picture coding systems, the basic construction of a coding system using the motion-compensated adaptive prediction and the discrete cosine transform is shown in FIG. 1. In this figure, after an input image signal S1 is divided into a plurality of regions defined by a region divider 1, the motion-compensated adaptive prediction is carried out. That is, a motion-compensated adaptive predictor 2 detects a motion vector between the input image signal S1 and a reference image signal S2 of the previous frame, which is stored in a frame memory 3 and which has been already coded and locally decoded, and performs the motion compensation with respect to the reference image signal using the detected motion vector, so as to produce a prediction signal. However, in the motion-compensated adaptive predictor 2, a preferred prediction mode is selected from intraframe codings (prediction signal⊂0) directly using the motion compensated prediction and the input image signal S1, and the prediction signal S3 corresponding to the prediction mode is outputted.
Then, in a subtracter 4, the prediction signal S3 is subtracted from the input image signal S1 to output a predictive residual signal S4. With respect to each of blocks of a predetermined size, the discrete cosine transform (DCT) of the predictive residual signal S4 is carried out by means of a discrete cosine transformer 5. The DCT coefficient obtained by the discrete cosine transform is quantized by means of a quantizer 6. The DCT coefficient quantized by the quantizer 6 is divided into two portions. One of the two portions is coded by means of a variable-length encoder 7, and then, it is multiplexed with the motion vector, which has been coded by a variable-length encoder 9, by means of a multiplexer 8 to be outputted as a bit-stream. The other portion is inverse-quantized by means of a inverse quantizer 10, and then, the inverse discrete-cosine transform (inverse DCT) thereof is carried out by means of an inverse discrete-cosine transformer 11. The output of the inverse discrete-cosine transformer 11 is added to the adaptive prediction signal S3 by means of an adder 12 to be a locally decoded signal to be stored in the frame memory 3.
FIG. 2 is a view illustrating the basic construction of a moving-picture decoding system which corresponds to the moving-picture coding system of FIG. 1. The code string transmitted from the moving-picture coding system to be stored is divided into a quantized DCT coefficient and a motion vector data by means of a demultiplexer 13. The quantized DCT coefficient data passes through a variable-length decoder 14, a inverse quantizer 15 and a inverse discrete-cosine transformer 16, to be outputted as a predicted error signal S6. The motion vector data is decoded by means of a variable-length decoder 17, and then, it is inputted to a motion-compensation predictor 18. In the motion-compensation predictor 18, the motion compensation to a reference image signal S7 of the last frame in a frame memory 19 is carried out using the motion vector to produce a prediction signal S8. Then, in an adder 20, the predicted error signal S6 is added to the prediction signal S8 to reproduce an image signal S9. The reproduced image signal S9 is outputted to the outside of the system and stored in the frame memory 19 as the reference image signal S7.
However, in such a conventional moving-picture coding and/or decoding system, there are the following problems.
In a channel in which an error may be mixed, such as a radio channel, when only the aforementioned coding is carried out, the quality of the decoded image is remarkably deteriorated if an error occurs. In particular, when there is an error in a signal such as a synchronizing signal, a mode data and a motion vector, the picture quality is remarkably deteriorated.
In addition, as mentioned above, the motion-compensated adaptive prediction coding is frequently used in the moving picture coding. However, since only the interframe difference is coded in the motion-compensated adaptive prediction coding, when an error occurs, the frame is not only incorrect, but an incorrect image is also stored- in a frame memory, so that a predicted image is prepared using the incorrect image and a residual error is added thereto. For that reason, even if the subsequent frame is correctly decoded, it is not possible to obtain a correctly decoded image in the subsequent frame, except that when the data is transmitted in a mode (INTRA mode) wherein the coding is performed in only the frame without using the interframe difference or when the influence of the error is gradually attenuated to return to the original state.
FIG. 3 illustrates this condition. This example shows the condition that a black circle is moving. Signals including a residual signal representative of a circle of the next frame (expressed by the black circle of the residual signal) and a residual signal for erasing a circle of the last frame (expressed by a dotted-line circle of the residual signal) are outputted as interframe differential signals. In this case, the motion compensation is not performed for simplification, and the interframe difference is obtained assuming that MV (motion vector)=0.
If data of one frame is lost due to error, the second frame is not decoded at all, and for example, the first frame is outputted as it is. In the third frame, a residual error, which is capable of correctly decoding if it is added to the second frame, is added to the first frame to reproduce a quite different image. Thereafter, since the residual error is added to the incorrect image, the error is not basically erased, so that it is not possible to reproduce a correctly decoded image.
In order to eliminate the aforementioned problem, a method called xe2x80x9crefreshxe2x80x9d for coding in the INTRA mode every a predetermined cycle has been conventionally used. In this case, when the coding is performed in the INTRA mode, the code amount is increased to remarkably deteriorate the picture quality when no error occurs. Therefore, methods such as the cyclic refresh for refreshing every few macroblocks in one frame, not simultaneously refreshing the whole picture, are generally used. However, although -the cyclic refresh is able to resstring the increase of the code string, there is a problem in that it takes a long time until the normal state is restored.
As other measures against errors, there is the use of the error correction coding. The error correction coding is able to correct errors caused at random. However, if errors of hundreds bits occur at a burst and continuously, it is difficult to correct such errors. Even if it is possible to correct such errors, a very long redundancy is required.
As mentioned above, in an image coding, particularly in a moving-picture coding, the loss of data due to error greatly deteriorates the picture quality. In addition, in the conventional methods such as the cyclic refresh for restoring the lost data due to error, there are problems in that it takes a long time until it is restored considering the coding efficiency, and that the attempt to decrease the time required to restore the lost data increases the code amount to lower the efficiency.
It is therefore an object of the present invention to provide a moving-picture coding and/or decoding system which can quickly restore if data is lost due to error and in which the increased code amount is less than those of the refresh and the error correction.
In order to accomplish the aforementioned and other objects, according to one aspect of the present invention, a moving-picture decoding system comprises: coding means for coding an input image signal to output a basic code string; code-string delay means for delaying the basic code string for a predetermined period of time to output as an additional code string; and code-string combining means for combining the basic code string with the additional code string to output the synthesized code string.
According to another aspect of the present invention, a moving-picture decoding system corresponding to the aforementioned moving-picture coding system, comprises: code-string dividing means for dividing an input code string into a basic code string and an additional code string obtained by delaying the basic code string; decoding means for decoding the basic code string or the additional code string to output a decoded data; error discriminating means for discriminating whether it is possible to decode the basic code string from the decoded data by the decoding means; and code-string switching means for inputting the basic code string to the decoding means when it is discriminated by the discriminating means that it is possible to decode the basic code string by the decoding means, and for inputting the additional code string to the decoding means when it is discriminated by the discriminating means that it is impossible to decode the basic code string by the decoding means.
Thus, according to the present invention, after the code string obtained by coding the input image signal is outputted as the basic code string, the additional code string which is basically the same as the basic code string is outputted again after a predetermined period of time. Therefore, even if the first outputted data of the basic code string is destroyed due to error during the transmission/storing, it is compensated by the data of the additional outputted after a predetermined period of time, so that the decoding is correctly performed.
In addition, according to the present invention, an additional code string simplified by selecting only important data and so forth may be outputted to reduce the code amount, in place of the directly outputting of the same additional code string as the first outputted basic code string. The image data decoding system includes interpolating means for interpolating the parts of the additional code string simplified to reduce the code amount, and decodes the additional code string after the interpolation when it is discriminated that it is impossible to decode the basic code string. Thus, it is possible to improve the error resilience without remarkably deteriorating the decoded image when no error occurs.
Moreover, according to the present invention, a synchronizing signal may be added to the additional code string in the moving-picture coding system so as to form one frame by only the additional code string. In this case, the moving-picture decoding system discriminates the basic code string from the additional code string by the synchronizing signal added to the additional code string, and divides the input code string into the basic code string and the additional code string on the basis of the discriminated results. Thus, since the moving-picture decoding system can discriminate the basic code string from the additional code string by only detecting the synchronizing signal, its construction is simple. In addition, if the synchronizing signal is thus added to the additional code string, the total number of the synchronizing signals increases, so that the opportunity for restoring the synchronism increases.
As mentioned above, according to the present invention, it is possible to provide a moving-picture coding and/or decoding system, which can quickly restore if data is lost due to error caused during the transmission/storing, in which the deterioration of quality of the decoded image is small, in which the increase of the code amount is smaller than those in the cyclic refresh and the error correction which have been conventionally performed as the measures against errors, and which has a high coding efficiency.
That is, according to the present invention, after a code string produced by coding an input image signal is outputted as a basic code string, an additional code string having basically the same contents is outputted again after a predetermined period of time, so that it is possible to perform the correct decoding using the data of the additional code string even if the data of the basic code string is destroyed due to errors caused during the transmission/storing.
In addition, if an additional code string simplified by selecting only important data from the basic code string and so forth to reduce the code amount, is outputted as the additional code string to interpolate the simplified parts of the additional code string on the decoding side for decoding, it is possible to further improve the error resilience without remarkably deteriorating the quality of the decoded image when no error occurs.
Moreover, a synchronizing signal may be added to the additional code string to form one frame by only the additional code string, so that the basic code string may be discriminated from the additional code string by the synchronizing signal added to the additional code string on the decoding side so as to divide the input code string into the basic code string and the additional code string. Thus, the construction is simple and the total number of the synchronizing signals increases, so that the opportunity for restoring the synchronism. Therefore, it is possible to further improve the error resilience.